Skip to content

Assertions#67

Merged
calebsyring merged 14 commits into
masterfrom
tim/assertion
Mar 3, 2026
Merged

Assertions#67
calebsyring merged 14 commits into
masterfrom
tim/assertion

Conversation

@tgubler9
Copy link
Copy Markdown
Collaborator

@tgubler9 tgubler9 commented Feb 9, 2026

Implementation of #64

Goals

  • Assertions are a string of the form <Assertion Subject> <operator> <valid expected data>
  • This string must be parsed correctly from DDB to Assertion objects
  • The object must be serializable from an Assertion object to a ddb string
  • String after being parsed must be able to correctly run the operation
  • Failure during run_checks should give a reason why the assertion failed

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 9, 2026

Codecov Report

❌ Patch coverage is 91.25683% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.67%. Comparing base (6b52ea8) to head (90d5735).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/critic/libs/assertions.py 86.84% 6 Missing and 4 partials ⚠️
src/critic/libs/uptime.py 83.33% 1 Missing and 2 partials ⚠️
tests/critic_tests/test_assertions.py 95.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #67      +/-   ##
==========================================
- Coverage   92.29%   91.67%   -0.62%     
==========================================
  Files          15       17       +2     
  Lines         662      829     +167     
  Branches       44       60      +16     
==========================================
+ Hits          611      760     +149     
- Misses         46       57      +11     
- Partials        5       12       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tgubler9 tgubler9 changed the title basic tests passing for pydantic validation of assertions Assertions Feb 10, 2026
@tgubler9
Copy link
Copy Markdown
Collaborator Author

tgubler9 commented Feb 10, 2026

Failing because tests of evaluate method have not been written yet

@tgubler9 tgubler9 self-assigned this Feb 10, 2026
@tgubler9 tgubler9 added the enhancement New feature or request label Feb 10, 2026
@tgubler9 tgubler9 marked this pull request as ready for review February 16, 2026 17:15
@tgubler9
Copy link
Copy Markdown
Collaborator Author

Should be good, but I need to add more tests

Comment thread src/critic/libs/assertions.py Outdated
Comment thread src/critic/libs/assertions.py Outdated
Comment thread src/critic/libs/assertions.py Outdated
Copy link
Copy Markdown
Owner

@calebsyring calebsyring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I have a few minor / nonstructural things to take a look at

Comment thread src/critic/libs/assertions.py Outdated
actual = response.text
elif self.assertion_object == AssertionSubject.RESPONSE_TIME:
actual = response.elapsed.total_seconds() * 1000

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add an else here with an error for unhandled cases

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldnt this be dealt with by the cast function?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but if we update the cast function and forget to update this function we might end up with an odd error. Not a big deal either way.

Comment thread src/critic/libs/assertions.py Outdated
Comment thread src/critic/libs/uptime.py Outdated
Makes the request and returns the response and the time it took to make the request.
"""

# TODO figure out if we want to use time.perf_counter or built in httpx response latency
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably do want to use time.perf_counter so that we still have a latency value available if there is a timeout exception.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tgubler9 we can remove this comment now right?

Comment thread src/critic/libs/uptime.py Outdated
slug='my-monitor',
url='https://example.com/health',
)
# Pretend we've received data via the API
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you decide to stop using the factories here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dumb mistake 🤷‍♂️

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method still needs to be switched back to the factory I think.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oop, switched a different one

Comment thread tests/critic_tests/test_libs/test_uptime.py
Comment thread pyproject.toml Outdated
Comment on lines +21 to +22
# Manually set the elapsed time to 500ms
resp.elapsed = timedelta(milliseconds=20.1)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment doesn't match elapsed. Also, this function isn't used anywhere

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not sure what you mean its not used anywhere, elapsed is used for the response time in the evaluate method.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean custom_response isn't called anywhere as far as I can tell.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ``httpx_mock` is not being used you are correct, so I can just get rid of that. The resp is still being used, and it worked without that fixture as far as I know

Comment thread src/critic/models.py
@calebsyring calebsyring merged commit e54cb68 into master Mar 3, 2026
3 of 5 checks passed
@calebsyring calebsyring deleted the tim/assertion branch March 3, 2026 15:28
@calebsyring calebsyring mentioned this pull request Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants